Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build System Rework #781

Open
wants to merge 195 commits into
base: main
Choose a base branch
from
Open

Build System Rework #781

wants to merge 195 commits into from

Conversation

dudoslav
Copy link
Contributor

@dudoslav dudoslav commented Oct 17, 2024 •
edited
Loading

This PR reworks the build system in order to properly build and package TileDB-VCF for all platforms (Linux, MacOS) and all package managers (PyPI and Conda).

The main goals of this PR:

  • Divide build into TileDB-VCF and TileDB-VCF-Py
  • TileDB-VCF needs to build as shared object and as a static library
  • TileDB-VCF needs to internally link with shared libraries or static libraries (fmt, spdlog, cli11)
  • TileDB-VCF-Py needs to properly package for PyPI, currently statically linked wheel
  • TileDB-VCF-Py conda packaging logic, only build TileDB-VCF-Py and dynamically link to TileDB-VCF (that also uses shared libs)

Windows build and package does not work yet.

dudoslav and others added 30 commits August 28, 2024 10:13
@dudoslav
Add initial build wheels CI/CD pipeline
ad3f0aa
@dudoslav
Fix working directory for cibuildwheels
486a255
@dudoslav
Fix incorrect package-dir option
d206d82
@dudoslav
Disable all platforms except Ubuntu
d478e19
@dudoslav
Install zlib1g-dev
963f3bd
@dudoslav
Fix install deps command
7c3c756
@dudoslav
Install xz as well
85c4afd
@dudoslav
Use BEFORE_ALL instead of BEFORE_BUILD
26d7a6a
@dudoslav
Update cibuildwheels version
33e30fc
@dudoslav
Enable macos and windows wheels
48ca0ee
@dudoslav
Install autoreconf for macos
9183194
@dudoslav
Add additional dependencies for macos build
469bc6c
@dudoslav
Disable macos
ba97a2b
@dudoslav
Change mingw htslib hash
aeae6ad
@dudoslav
Quick rework CMake files
47928e9
Fix widnows build
b0988c7
@dudoslav
Fix install group
1ded335
@dudoslav
Make tests optional
572d827
@dudoslav
Install tiledb.so
59df672
@dudoslav
Also install tiledbvcf
34bb75d
@dudoslav
Test wheels
ba57075
@dudoslav
Build with executables
019d843
@dudoslav
Fix macos workflow
2ab078a
@dudoslav
Modify macos workflow
e15b381
@dudoslav
Change linking check paths
be84c71
@dudoslav
Enable Macos wheel builds
9f36f87
@dudoslav
Fix macos dependency install
3c56c0d
@dudoslav
Install autoconf instead of autoreconf
4634884
@dudoslav
Change macosx deployment version
2b2f76d
@dudoslav
Enable sdist and testing
f72da21
@jdblischak jdblischak mentioned this pull request Dec 4, 2024
Open
@jdblischak jdblischak linked an issue Dec 4, 2024 that may be closed by this pull request
The new scikit-build-core setup copies external shared objects into Python wheel #717
Open
dudoslav added 23 commits December 9, 2024 15:01
@dudoslav
Properly handle install prefix subpaths
2852fa5
@dudoslav
Install module logic
7e34375
@dudoslav
For conda test, do not install do subfolder
91c4989
@dudoslav
Find in correct folder
bdf510e
@dudoslav
Test correct paths
4dd88fa
@dudoslav
Always install python bindings under tiledbvcf
7d4e3ed
@dudoslav
Set CMAKE_INSTALL_PREFIX before project
402655c
@dudoslav
Always install python bindings into tiledbvcf
5d34014
@dudoslav
Fix install paths
4ab918f
@dudoslav
Merge remote-tracking branch 'origin/main' into db/vcpkg
fe56081
@dudoslav
Macos CI/CD build options
0637a26
@dudoslav
Uncomment failing test
804acdc
@dudoslav
Uncomment java tests
9cc1025
@dudoslav
Fix nightly build
8fb35b3
@dudoslav
Gradle use cmake
ef7b816
@dudoslav
Install dependencies before building java bindings
9a68f0a
@dudoslav
Add debug logs
cedddcb
@dudoslav
Change find path for native lib for java
51e3337
@dudoslav
Merge remote-tracking branch 'origin/main' into db/vcpkg
41d7f4a
@dudoslav
Fix java bindings find_library
415fff8
@dudoslav
Fix include paths for java
c69c63b
@dudoslav
Fix find native lib
da25ebb
@dudoslav
Merge remote-tracking branch 'origin/main' into db/vcpkg
0fc3bc9
gspowley
gspowley suggested changes Feb 21, 2025
View reviewed changes
Copy link
Contributor

@gspowley gspowley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for working on this @dudoslav. A couple of high level comments.

Please keep the Makefile in the root of the repo, which a convenience tool for developers as opposed to part of the build system.

Releasing the build system changes with pypi upload is more important than enabling windows support in this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@teo-tsirpanis teo-tsirpanis teo-tsirpanis left review comments

@ihnorton ihnorton ihnorton requested changes

@gspowley gspowley gspowley requested changes

Requested changes must be addressed to merge this pull request.

Assignees

@dudoslav dudoslav

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

The new scikit-build-core setup copies external shared objects into Python wheel
4 participants
@dudoslav @ihnorton @gspowley @teo-tsirpanis